Automated price maintenance for use with a system in which advertisements are rendered with relative preference based on performance information and price information

ABSTRACT

A cost maintenance scheme useful with an advertising system which orders ads in a manner that maximizes both their relevance and their economic value, and which provides a fair bidding process in which a winning bidder is assured of not having paid too much. Ordering may be done based on accepted maximum ad bid information and/or ad performance information. For example, this information may be used to determine a position value. Cost may be determined based on accepted maximum ad bid information and/or performance information. Billing the determined cost may be subject to a condition precedent.

§0. RELATED APPLICATION

Benefit is claimed, under 35 U.S.C. §119(e)(1), to the filing date ofprovisional patent application Ser. No. 60/424,792, entitled “AUTOMATEDPRICE MAINTENANCE FOR USE WITH A SYSTEM IN WHICH ADVERTISEMENTS ARERENDERED WITH RELATIVE PREFERENCE BASED ON PERFORMANCE INFORMATION ANDPRICE INFORMATION”, filed on Nov. 8, 2002 and listing Eric Veach as theinventor, for any inventions disclosed in the manner provided by 35U.S.C. §112, ¶ 1. That provisional application is expressly incorporatedherein by reference.

§1. BACKGROUND OF THE INVENTION

§1.1 Field of the Invention

The present invention concerns advertising. In particular, the presentinvention concerns how ads are to be presented to their audience and howadvertisers can manage their advertising costs.

§1.2 Related Art

Advertising using traditional media, such as television, radio,newspapers and magazines, is well known. Advertisers have used thesetypes of media to reach a large audience with their advertisements(“ads”). To reach a more responsive audience, advertisers have useddemographic studies. For example, advertisers may use broadcast eventssuch as football games to advertise beer and action movies to a youngermale audience. Similarly, advertisers may use magazines that reach arelatively affluent readership to advertise luxury items such asexpensive watches and luxury automobiles. However, even with demographicstudies and entirely reasonable assumptions about the typical audienceof various media outlets, advertisers recognize that much of their adbudget is simply wasted. Unfortunately, it is very difficult to identifyand eliminate such waste.

Recently, advertising over more interactive media has become popular.For example, as the number of people using the Internet has exploded,advertisers have come to appreciate media and services offered over theInternet as a potentially powerful way to advertise.

Advertisers have developed several strategies in an attempt to maximizethe value of such advertising. In one strategy, advertisers use popularpresences or means for providing interactive media or services (referredto as “Web sites” in the specification without loss of generality) asconduits to reach a large audience. Using this first approach, anadvertiser may place ads on the home page of the New York Times Website, or the USA Today Web site, for example. In another strategy, anadvertiser may attempt to target its ads to more narrow niche audiences,thereby increasing the likelihood of a positive response by theaudience. For example, an agency promoting tourism in the Costa Ricanrainforest might place ads on the ecotourism-travel subdirectory of theYahoo Web site.

Regardless of the strategy, Web site-based ads (also referred to as “Webads”) are typically presented to their advertising audience in the form“banner ads”—i.e., a rectangular box that includes graphic components.When a member of the advertising audience (referred to as a “viewer” inthe Specification without loss of generality) selects one of thesebanner ads by clicking on it, embedded hypertext links typically directthe viewer to the advertiser's Web site. This process, wherein theviewer selects an ad, is commonly referred to as a “click-through”.(“Click-through” is intended to include any user selection.) The ratioof the number of click-throughs to the number of impressions of the ad(i.e., the number of times an ad is displayed) is commonly referred toas the “click-through rate” of the ad.

Despite the initial promise of Web site-based advertisement, thereremain several problems with existing approaches. Although advertisersare able to reach a large audience, they are frequently dissatisfiedwith the return on their advertisement investment. Some have attemptedto improve ad performance by tracking the online habits of users, butthis approach has led to privacy concerns.

Similarly, the hosts of Web sites on which the ads are presented(referred to as “Web site hosts” or “ad consumers”) have the challengeof maximizing ad revenue without impairing their users' experience. SomeWeb site hosts have chosen to place advertising revenues over theinterests of users. One such Web site is “Overture.com”, which hosts aso-called “search engine” service returning purported “search results”in response to user queries. The Overture.com web site permitsadvertisers to pay to position an ad for their Web site (or a target Website) higher up on the list of search results. If such schemes in whichthe advertiser only pays if a user clicks on the ad (i.e.,cost-per-click) are implemented, the advertiser lacks incentive totarget their ads effectively, since a poorly targeted ad will not beclicked and therefore will not require payment. As a result, highcost-per-click ads show up near or at the top, but do not necessarilytranslate into real revenue for the ad publisher because viewers don'tclick on them. Furthermore, ads that viewers would click on are furtherdown the list, or not on the list at all, and so relevancy of ads iscompromised.

U.S. patent application Ser. No. 10/112,654, entitled “METHODS ANDAPPARATUS FOR ORDERING ADVERTISEMENTS BASED ON PERFORMANCE INFORMATIONAND PRICE INFORMATION”, filed on Mar. 29, 2002, and listing SalarKamangar, Eric Veach and Ross Koningstein as the inventors (hereafterreferred to as “the Kamangar application”, and incorporated herein byreference.), provides a better scheme, in which ads are positioned (orotherwise rendered with relative preference) as a function of both priceand at least one performance parameter (such as click-through rate forexample).

It would be useful to help advertisers manage their bidding in any ofthe foregoing schemes (that is, the foregoing schemes in which adposition (or other relative preferential rendering) is, at least inpart, based on a bid price). Software which automates tracking bids,bidding, and updating bids (known as “robots”) are known. However, suchrobots allow sophisticated bidders to have an unfair advantage overothers. That is, sophisticated bidders can repeatedly win a bid byincreasing their bid by the minimum increment (e.g., $0.01) over thehighest bid automatically and without exposing the maximum that they arewilling to bid. Therefore, it would be desirable to make the biddingprocess more fair, while permitting a winning bidder to avoid “winner'sremorse” (i.e., allowing the winner to pay the least amount of money tomaintain the position (or other relative rendering preference of theirad).

§2. SUMMARY OF THE INVENTION

The present invention is useful with an advertising system whichprovides a fair bidding process in which a winning bidder is assured ofnot having paid too much. Ads may be ordered based on accepted maximumad bid information, or a combination of maximum ad bid information andad performance information. For example, this information may be used todetermine a position (or some other ad preference) value. Cost may bedetermined based on the accepted maximum ad bid information and the nextlower position value.

The performance information is generally a measure of user interest inthe associated advertisement and may be, for example, (a) aclick-through rate of the associated advertisement, (b) user ratings ofthe advertisement, (c) focus group ratings of the advertisement, (d) ameasure of user interest for the advertisement weighted for a size ofthe advertisement relative to that of other advertisements, (e) ameasure of user interest for the advertisement weighted for pastpositions of the advertisement relative to those past positions of otheradvertisements, (f) expected user interest in the advertisement, (g) atime needed to render the advertisement relative to that needed torender other advertisements, (h) a measure of user interest for theadvertisement weighted for a media type of the advertisement, or (i) aconversion rate associated with the advertisement. Such performanceinformation may be weighted, windowed, and/or averaged.

§3. BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a high-level diagram showing parties or entities that caninteract with an advertising system.

FIG. 2 is a bubble chart of an advertising environment in which, or withwhich, the present invention may operate.

FIG. 3 is a bubble chart of some accounting and billing operations thatmay be used in the advertising environment of FIG. 2.

FIG. 4 is a block diagram of a Web page that may be generated by a pageassembly operation of an ad consumer, for rendering on a viewer'sscreen.

FIG. 5 is a flow diagram of an exemplary method that may be used toeffect a presentation ordering operation in an environment such as thatof FIG. 2.

FIG. 6 is a table of values in an example which illustrates how adordering and cost determination may be performed.

FIG. 7 is a flow diagram of an exemplary method that may be used todetermine a cost to be charged to an advertiser.

FIG. 8 is a flow diagram of an exemplary method that may be used to billan account for an ad (to be) served.

FIG. 9 is a high-level block diagram of apparatus that may be used toeffect at least some of the various operations that may be performed inaccordance with the present invention.

FIG. 10 illustrates a relationship among ad identification information,keyword(s), ad content, a maximum cost per result bid and an averagecost per result bid.

FIG. 11 is a messaging diagram illustrating exemplary operations of afirst exemplary embodiment of the present invention.

FIG. 12 is a messaging diagram illustrating exemplary operations of asecond exemplary embodiment of the present invention.

§4. DETAILED DESCRIPTION

The present invention may involve novel methods, apparatus, messageformats and/or data structures for providing effective advertisements inan interactive environment, for providing a fair bidding process, andfor protecting a winning bidder from paying too much. The followingdescription is presented to enable one skilled in the art to make anduse the invention, and is provided in the context of particularapplications and their requirements. Various modifications to thedisclosed embodiments will be apparent to those skilled in the art, andthe general principles set forth below may be applied to otherembodiments and applications. Thus, the present invention is notintended to be limited to the embodiments shown and the inventors regardtheir invention as any patentable subject matter described.

In the following, environments in which the present invention mayoperate are described in 4.1. Then, exemplary embodiments of the presentinvention are described in 4.2. Examples of operations of an exemplaryembodiment of the invention is then provided in 4.3. Finally, someconclusions regarding the present invention are set forth in 4.4.

§4.1 Environments in which, or with which, the Present Invention MayOperate

§4.1.1 Exemplary Advertising Environment

FIG. 1 is a high level diagram of an advertising environment. Theenvironment may include an ad entry, maintenance and delivery system120. Advertisers 110 may directly, or indirectly, enter, maintain, andtrack ad information in the system 120. The ads may be in the form ofgraphical ads such as so-called banner ads, text only ads, audio ads,video ads, etc. Ad consumers 130 may submit requests for ads to, acceptads responsive to their request from, and provide historical or usageinformation to, the system 120. Although not shown, other entities mayprovide historical or usage information (e.g., whether or not aconversion or click-through related to the ad occurred) to the system120.

One example of an ad consumer 130 is a general content server whichreceives requests for content (e.g., articles, discussion threads,music, video, graphics, search results, web page listings, etc.), andretrieves the requested content in response to, or otherwise services,the request. The content server may submit a request for ads to thesystem 120. Such an ad request may include a number of ads desired. Thead request may also include content request information. Thisinformation may include the content itself (e.g., page), a categorycorresponding to the content or the content request (e.g., arts,business, computers, arts-movies, arts-music, etc.), part or all of thecontent request, content age, content type (e.g., text, graphics, video,audio, mixed media, etc.), geolocation information, etc.

The content server may combine the requested content with one or more ofthe advertisements provided by the system 120. This combined informationincluding the content and advertisement(s) is then forwarded towards theend user that requested the content, for presentation to the viewer.Finally, the content server may transmit information about the ads andhow the ads are to be rendered (e.g., position, click-through or not,impression time, impression date, size, conversion or not, etc.) back tothe system 120. Alternatively, or in addition, such information may beprovided back to the system 120 by some other means.

Another example of an ad consumer 130 is a search engine. A searchengine may receive queries for search results. In response, the searchengine may retrieve relevant search results (e.g., from an index of Webpages). An exemplary search engine is described in the article S. Brinand L. Page, “The Anatomy of a Large-Scale Hypertextual Search Engine,”Seventh International World Wide Web Conference, Brisbane, Australia andin U.S. Pat. No. 6,285,999 (both incorporated herein by reference). Suchsearch results may include, for example, lists of Web page titles,snippets of text extracted from those Web pages, and hypertext links tothose Web pages, and may be grouped into a predetermined number of(e.g., ten) search results.

The search engine may submit a request for ads to the system 120. Therequest may include a number of ads desired. This number may depend onthe search results, the amount of screen or page space occupied by thesearch results, the size and shape of the ads, etc. In one embodiment,the number of desired ads will be from one to ten, and preferably fromthree to five. The request for ads may also include the query (asentered or parsed), information based on the query (such as geolocationinformation, whether the query came from an affiliate and an identifierof such an affiliate), and/or information associated with, or based on,the search results. Such information may include, for example,identifiers related to the search results (e.g., document identifiers or“docIDs”), scores related to the search results (e.g., informationretrieval (“IR”) scores such as dot products of feature vectorscorresponding to a query and a document, Page Rank scores, and/orcombinations of IR scores and Page Rank scores), snippets of textextracted from identified documents (e.g., WebPages), full text ofidentified documents, feature vectors of identified documents, etc.

The search engine may combine the search results with one or more of theadvertisements provided by the system 120. This combined informationincluding the search results and advertisement(s) is then forwardedtowards the user that requested the content, for presentation to theuser. For example, FIG. 4 is an abstract illustration of a display page410 that may be provided by the search engine. The outline 420 depictedwith dashed lines corresponds to a portion of the display page 410 thatmay be viewed on a typical personal computer display screen at a typicalresolution. The exemplary display page 410 may include headerinformation 412 (e.g., the name of search engine host), trailerinformation 416 (e.g., copyright, navigational hypertext links, etc.), aplurality of search results 414 and a plurality of ads 418 a, 418 b, and418 c. Preferably, the search results 414 are maintained as distinctfrom the ads 418, so as not to confuse the user between paidadvertisements and presumably neutral search results. Although FIG. 4shows only three ads 418, embodiments consistent with the presentinvention may have more or less ads. For example, ten search resultscombined with ten ads has been found to be effective.

Finally, the search engine may transmit information about the ad and howthe ad was to be rendered (e.g., position, click-through or not,impression time, impression date, size, conversion or not, etc.) back tothe system 120. Alternatively, or in addition, such information may beprovided back to the system 120 by some other means.

§4.1.2 Exemplary Ad Entry, Maintenance and Delivery Environment

FIG. 2 illustrates an exemplary ad system 120′, consistent with thepresent invention. The exemplary ad system 120′ may include an inventorysystem 210 and may store ad information 205 and usage or historical(e.g., statistical) information 245. The exemplary system 120′ maysupport ad information entry and management operation(s) 215, campaign(e.g., targeting) assistance operation(s) 220, accounting and billingoperation(s) 225 (which may include a cost determination operation(s)227), ad serving operation(s) 230, relevancy determination operation(s)235, optimization operations 240, presentation ordering operations 250,fraud detection operation(s) 255, and result(s) interface operation(s)260. Advertisers 110 may interface with the system 120′ via the adinformation entry and management operation(s) 215 as indicated byinterface 216. Ad consumers 130 may interface with the system 120′ viathe ad serving operation(s) 230 as indicated by interface 231. Adconsumers 130 or other entities (not shown) may also interface with thesystem 120′ via results interface operation(s) 260 as indicated byinterface 261.

Referring to FIG. 3, the accounting/billing operations 225 may includecost determination operation(s) 227 for generating determined costinformation 229, as well as billing operations 228 that use suchdetermined cost information. Notice that the ad serving operation(s) 230may also use the determined cost information 229 (as stored, or asprovided directly from the cost determination operation(s). One aspectof the present invention concerns the cost determination operation(s)227. Various aspects of the cost determination operation(s) 227 maydepend on techniques used in the presentation ordering operations 250.The cost determination operation(s) 227 is described more fully insection 4.2.

An advertising program may include information concerning accounts,campaigns, creatives, targeting, etc. The term “account” relates toinformation for a given advertiser (e.g., a unique email address, apassword, billing information, etc.). A “campaign” or “ad campaign”refers to one or more groups of one or more advertisements, and mayinclude a start date, an end date, budget information, geo-targetinginformation, syndication information, etc. For example, Honda may haveone advertising campaign for its automotive line, and a separateadvertising campaign for its motorcycle line. The campaign for itsautomotive line have one or more ad groups, each containing one or moreads. Each ad group may include a set of keywords, and a maximum cost bid(cost per click-though, cost per conversion, etc.). Alternatively, or inaddition, each ad group may include an average cost bid (e.g., averagecost per click-through, average cost per conversion, etc.). Therefore, asingle maximum cost bid and/or a single average cost bid may beassociated with one or more keywords. As stated, each ad group may haveone or more ads or “creatives” (That is, ad content that is ultimatelyrendered to an end user.).

The ad information 205 can be entered and managed via the ad informationentry and management operation(s) 215. Campaign (e.g., targeting)assistance operation(s) 220 can be employed to help advertisers 110generate effective ad campaigns. The campaign assistance operation(s)220 can use information provided by the inventory system 210, which, inthe context of advertising for use with a search engine, may track allpossible ad impressions, ad impressions already reserved, and adimpressions available for given keywords. The ad serving operation(s)230 may service requests for ads from ad consumers 130. The ad servingoperation(s) 230 may use relevancy determination operation(s) 235 todetermine candidate ads for a given request. The ad serving operation(s)230 may then use optimization operation(s) 240 to select a final set ofone or more of the candidate ads. Finally, the ad serving operation(s)230 may use presentation ordering operation(s) 250 to order thepresentation of the ads to be returned. The fraud detection operation(s)255 can be used to reduce fraudulent use of the advertising system(e.g., by advertisers), such as through the use of stolen credit cards.Finally, the result(s) interface operation(s) 260 may be used to acceptresult information (from the ad consumers 130 or some other entity)about an ad actually served, such as whether or not click-throughoccurred, whether or not conversion occurred (e.g., whether the sale ofan advertised item or service was initiated or consummated within apredetermined time from the rendering of the ad), etc. Such result(s)information may be accepted at interface 261 and may include informationto identify the ad and time the ad was served, as well as the associatedresult.

Recall that various aspects of the cost determination operation(s) 227may depend on techniques used in the presentation ordering operations250. FIG. 5 is a flow diagram of an exemplary method 250′ that may beused to effect such presentation ordering operation(s). This exemplarymethod corresponds to the method described in the Kamangar application.As indicated by block 510, a set (e.g., a list) of candidate ads isobtained. Referring to the Table of FIG. 6, suppose that this listcontains the five ads—A, B, C, D, and E. Then, as indicated by block520, one or more performance parameters (or more generally, “performanceinformation”) for each candidate ad is identified. In oneimplementation, the performance parameter is a windowed, time-weightedaverage click-through rate for the ad. As shown in the Table of FIG. 6,assume that ads A, B, C, D, and E have the following windowed,time-weighted average click-through rates, respectively: 1%, 4%, 4%, 3%and 12%.

Similarly, as indicated by block 530, a price parameter (or moregenerally, “price information”) is identified for each candidate ad.Examples of a price parameter include a maximum cost-per-impression bid,a maximum cost-per-selection (e.g., click-through) bid, a maximumcost-per-conversion bid, etc. In the example illustrated in FIG. 6, thisprice is a maximum cost-per-click bid, which is preferably defined inadvance as a negotiated or auction-based bid. As shown in the Table ofFIG. 6, assume that at a given point in time, ads A, B, C, D, and E havethe following maximum costs-per-click bids, respectively: $5.00 perclick, $1.50 per click, $1.00 per click, $0.90 per click, and $0.25 perclick.

In the exemplary method 250′ of FIG. 5, as indicated by block 540, aplacement value (or more generally, “an ad preference value”) isdetermined for each candidate ad based on the one or more performanceparameters and the price. In one implementation, this placement value isa product of the windowed, time-weighted average click-through rate andthe maximum cost-per-click bid. Accordingly, as shown in the Table ofFIG. 6, ad A has a score of 0.050 (0.01 multiplied by 5.00), ad B has ascore of 0.060 (0.04 multiplied by 1.50), ad C has a score of 0.040(0.04 multiplied by 1.00), ad D has a score of 0.027 (0.03 multiplied by0.90), and ad E has a score of 0.030 (0.12 multiplied by 0.25). Finally,as indicated by block 550, the ads may be ordered based on theirplacement values, and the method 250′ may then be left via RETURN node560. In the present example, the order would be B, A, C, E, D.

As described in the Kamangar application, in certain cases, it may bedesirable to modify the placement value produced by the method describedby FIG. 5, to take into account unique information. For example, it maybe desirable to increase the ad scores of certain valued or strategicadvertisers, increase the placement values for campaigns that are timesensitive, etc. There may, of course, exist other reasons to increase ordecrease the placement values for particular campaigns, ads,advertisers, etc. Such increases or decreases may be achieved byemploying a multiplier to the overall resulting placement values,individual components of the score (e.g., click-through rate, maximumcost-per-click bid, etc), etc. Other functions, including linearfunctions, polynomial functions, and exponential functions for example,may employ such coefficients or values to adjust placement values.Statistical weighting (e.g., based on a deviation such as a standarddeviation) may also be used to adjust price and/or performanceinformation. Such time and/or statistical weighting may be used, forexample, to desensitize the presentation ordering operation(s) to spikesor other anomalies.

Using the example page shown in FIG. 4, assume that the ad consumer 130is a search engine that requested three ads. Using the method describedin reference to FIG. 5, the three highest scoring ads (B, A, and C)would be returned, with ad B being shown in position 418 a, ad A inposition 418 b, and ad C in position 418 c. In an alternativeembodiment, a relative size or other prominence feature of the ad, inaddition to, or instead of, the placement order of the ads, may be basedon the placement values (or the ad preference value) associated with theads.

As described in the Kamangar application, it may be desirable not toshow only the highest ranking ads. For example, it may be that new orlow ranking ads have not been shown enough to have a statisticallymeaningful performance parameter. In this case (or even in general), onemight artificially or temporarily increase the placement values forcertain ads (e.g., new or low ranking ads) at random, periodic, or otherintervals. Alternatively, it may be desirable never to return ads withscores below a defined threshold, so long as they have been shown astatistically sufficient number of times to discern their performance.

Note that the exemplary ordering method 250′ is advantageous in that itcan be used to “normalize” cost per different types of result anddifferent performance parameters to some monetary or revenue unit. Forexample, if a first advertiser wanted to pay based on conversions, whilea second advertiser wanted to pay based on click-throughs, the placementvalues could be normalized to dollars as shown:

${\left. {\frac{dollars}{conversion} \times \frac{conversions}{100\mspace{14mu}{impressions}}}\Rightarrow\frac{dollars}{100\mspace{14mu}{impressions}} \right.;{and}}\mspace{11mu}$$\;\left. {\frac{dollars}{{click}\text{-}{through}} \times \frac{{click}\text{-}{throughs}}{100\mspace{14mu}{impressions}}}\Rightarrow\frac{dollars}{100\mspace{14mu}{impressions}} \right.$Such normalization permits advertisers that want to pay for theiradvertising using different metrics to compete on a common basis.

Using conversion rate information may be particularly attractive toadvertisers, for it provides a direct way for advertisers to comparetheir advertising cost to the benefits they obtain from thatadvertisement. For example, suppose an advertiser (Advertiser W) is ableto estimate that approximately 15% of users who see an advertisement fora set of keywords ultimately make a purchase from Advertiser W. Andsuppose further that Advertiser W can estimate that on average each suchpurchase resulted in revenue of $50 to Advertiser W. Advertiser W canthen determine that the value to it of each advertisement is $7.50 (0.15multiplied by $50). Advertiser W can use that information to decide howmuch it is willing to pay for those advertisements, which wouldpresumably be some amount less than $7.50. Indeed, Advertiser W couldeven provide ad system 120 with an average profit or revenue amount forconversions associated with a given set of keywords and leave it to adsystem 120 to maximize the benefit achieved by Advertiser W.

Suppose further that Advertiser X is able to estimate that for the sameset of keywords, approximately 30% of user who see its advertisementmake a purchase from Advertiser X for an average price of $50. The valueof the ad to Advertiser X is $15 (0.30 multiplied by $50). Therefore,Advertiser X ought to be willing to pay more for its advertisement thanAdvertiser W would be (e.g., up to $15 rather than up to $7.50).

An advertiser, however, may not want to provide ad system 120 with theconversion information specified above. The advertiser may, instead,simply specify the price they are willing to pay for each conversion.Using the example above, Advertiser X may specify a maximum (or average,or other) price per conversion of $15, while Advertiser W may specify aprice per conversion of $7.50. Ad system 120 may then use thatinformation in determining how to order the advertisements (e.g.,Advertiser X's ad first, then that of Advertiser W) and how much tocharge for those ads.

What constitutes a conversion may vary from case to case and can bedetermined in a variety of ways. For example, it may be the case that aconversion occurs when a user clicks on an ad, is referred to theadvertiser's web page, and consummates a purchase there before leavingthat web page. Alternatively, a conversion may be defined as a userbeing shown an ad, and making a purchase of the advertiser's web pagewithin seven days. Many other definitions of what constitutes aconversion are possible.

As mentioned earlier, a potential benefit of the organization ofexemplary display page 410 is that the search results 414 are maintainedas distinct from the ads 418. The search results are ordered based oncriteria that determines how relevant those information sources are tothe search query entered by the user. The ads are ordered based on themethods described above. Yet by maintaining clear linear separationbetween the search results 4614 and the ads 418, the user is not misledinto confusing search results with paid advertisements. This allows thedisplay of multiple ads on the same page as multiple search resultswhile maintaining separation between the two such that a viewer canclearly distinguish search results from paid ads. Although display page410 shows the advertisements positioned to the right of (adjacent)search results, other ways to separate the content from theadvertisements are also possible. For example, the advertisements couldbe positioned to the left of (adjacent) search results.

§4.2 EXEMPLARY EMBODIMENTS §4.2.1 Exemplary Cost Determination Methods

FIG. 7 is a flow diagram of an exemplary method 227′ that may be used toeffect a cost determination operation(s) 227. This exemplary method 227′ensures that each “winner” does not pay more than is necessary tomaintain the determined position (or other relative renderingpreference) of its ad. This method 227′ may be invoked, for example,after the ads that are to be served are determined and ordered, and ispreferably invoked for each ad (to be) served. As indicated by blocks710 and 720, for a given ad, the next lower placement value (or adpreference value) is accepted, and the performance parameter(s), used inthe determination of the placement value, is also accepted. Then, asindicated by block 730, a cost is determined based on the accepted nextlower placement (or ad preference) value and the accepted performanceparameter.

In one embodiment, the cost is determined simply by dividing the nextlower placement (or ad preference) value by the performance value(s).Continuing the example illustrated in the Table of FIG. 6 and started inconnection with the presentation ordering method 250′ of FIG. 5, thecost of placing ad B first is $1.25 (=ad A's placement value divided byad B's click-through rate=0.05/0.04), the cost of placing ad A second is$4.00 (=ad C's placement value divided by ad A's click-throughrate=0.04/0.01), the cost of placing ad C third is $0.75 (=ad E'splacement value divided by ad C's click-through rate=0.03/0.04), and thecost of placing ad E fourth is $0.225 (=ad D's placement value dividedby ad E's click-through rate=0.027/0.12).

In another embodiment of the invention, the cost charged for an ad maybe determined by dividing the next lower placement (or ad preference)value by the performance value(s), and adding an amount. The amount maybe a nominal value so that the ad with a higher placement (or adpreference) value costs at least the amount (or perhaps slightly morethan) necessary to render the ad in a higher placement or preferenceposition than the next lower-placed (or preferred) ad. For example, withrespect to the example illustrated in the Table of FIG. 6 above, onecent ($0.01) may be added to the cost per result billed to each of therendered ads B, A, and C, so that the cost of placing the ads in theirpreference order is ($1.25+$0.01)=$1.26 per click to place ad B first,($4.00+$0.01)=$4.01 to place ad A second, and ($0.75+$0.01)=$0.76 toplace ad C third. The amounts added to the ad costs for ads A, B, and Cneed not be the same, and may also be negative (such that, for example,the cost billed to ads B, A, and C may be ($1.25+$0.00)=$1.25,($4.00-$0.01)=$3.99, and ($0.75-$0.01)=$0.74, respectively). One ofordinary skill will further recognize that numerous additional orsubstitute methods may be employed to achieve the same effect, such asby rounding the cost per result billed to a nearest convenient increment(such as the nearest penny for transactions in United States dollars),adding an amount to the maximum bid cost used to determine preferenceorder, adjusting or rounding the performance parameter, etc.

The foregoing discussion of exemplary cost determination method 227′ isprovided with reference to a preferred system, such as disclosed in theKamangar application, in which ads are positioned (or otherwise renderedwith relative preference) as a function of both price and at least oneperformance parameter. However, one of ordinary skill in the art willrecognize that the present invention is not limited to such systems. Forexample, in other implementations, ads may be given a placement order(or rendering preference) as a function of price information alone(e.g., cost charged per click-through), without using a performanceparameter. Consistent with the invention, a cost determination methodensures that each advertiser does not pay more than is necessary tomaintain the relative rendering preference when ads are positioned inthis manner as a function of price but not performance.

In one embodiment, a cost (e.g. cost charged per click-through) for anad is determined based on a maximum bid (e.g., maximum cost perclick-through) of the next lower preferred ad. For example, the actualcost charged to an advertiser per click-through for the highest placed(or preferred) ad is determined based on the maximum bid perclick-through of the next lower ordered ad. With reference to Table ofFIG. 6 above, according to this embodiment of the invention, ads A, B,C, D, and E would have the placement order A, B, C, D, E (based on themaximum cost per click indicated by the advertiser and without referenceto any performance information). However, the actual cost (perclick-through) charged for ad A would be $1.50 (equal to the maximum bidper result of B, the next lower placed ad), the actual cost (perclick-through) charged for ad B would be $1.00 (equal to the maximum bidper result of C, the next lower placed ad), the actual cost (perclick-through) charged for ad C would be $0.90 (equal to the maximum bidper result of D, the next lower placed ad), the actual cost (perclick-through) charged for ad D would be $0.25 (equal to the maximum bidper result of E, the next lower placed ad), and the actual cost (perclick-through) charged for ad E would be set to some reserve price(e.g., $0.01).

As disclosed above, the actual costs charged to the advertisers whoseads are shown may be adjusted by adding an amount, such as $0.01, toprovide that a higher placed ad (e.g. A) pays an actual cost per resulthigher than the maximum bid per result of the next lower placed ad (e.g.B). Similarly, numerous other mechanisms exist for adjusting ormodifying the cost charged to advertisers.

The principle described above allows ad system 120 to separate theinformation used to order the ads from the actual cost charged. The adscan be ordered using solely price information (e.g., by maximum bid perclick), while the actual cost charged can be based on other priceinformation (e.g., the next lower maximum bid per click). Similarly, theads can be ordered using solely performance information (e.g., byconversion rate), while the actual cost charged can be based on priceinformation (e.g., bid amount, next lower bid amount, etc). Or the adscan be ordered using some combination of price and performanceinformation (e.g., maximum bid per click multiplied by click-throughrate), while the actual cost charged is based on price information(e.g., bid amount, next lower bid amount, etc).

Note that in each case, the advertiser usually pays a lower rate, e.g.the lowest rate, while still maintaining the position of its ad. Thecost of the lowest placed ad can be similarly determined, unless it didnot “beat out” another ad that was not served. In this case, the cost ofthe lowest placed ad can simply be some predetermined minimum cost(referred to as a “reserve price”), or some predetermined minimum costmodified by the performance parameter(s) of the ad. Although bids cancorrespond to multiple keywords (of a given ad or ad group), eachparticular keyword preferably has its own (system-wide) reserve price,though all keywords or groups of keywords can be associated with asingle reserve price.

In the case of an ad ordering system utilizing performance information,if the performance parameters have sufficient precision, ties inplacement (or ad preference) values should be rare. In the event of atie, such a tie may be broken arbitrarily, in favor of the ad with thehigher bid, in favor of the ad with the better performance parameter(s),based on some additional information or performance parameter notconsidered in determining the placement values, etc.

In one embodiment, both the ordering of the ads and the costs charged tothe advertisers are determined substantially in real time. In otherwords, when ad consumer 130 requests one or more advertisements fromsystem 120, system 120 determines the ads relevant to that request,determines the ordering of those ads, and determines the cost to becharged to those advertisers. Those skilled in the art will recognize,however, that other approaches may be used consistent with theinvention. For example, the actual cost to be charged to advertisers maybe determined at predefined or periodic intervals. Alternatively, it maybe more efficient to predetermine responses to certain requests (e.g.,popular search queries).

Once the cost is determined, as indicated by optional block 740, thedetermined cost (and related information) should be made available forbilling. Alternatively, or in addition, the determined cost can beencoded into the ad information served, as indicated by block 745. Thisencoded determined cost information may be encrypted and/or signed forsecurity purposes.

In one embodiment, any of the cost information used may be provided toadvertisers in order to allow advertisers to determine how to specifyinformation that best achieves their advertising goals. For example,maximum cost per result bid may be provided so that advertisers candetermine how they wish to bid relative to other advertisers' bids.Alternatively, or in addition, actual cost per result charged may beprovided. Similarly, performance information (average click-through orconversion rate, impressions/day, etc.), cost information (maximum bidcost, average billed cost per click or conversion, etc.) and resultsinformation (average billed amount per day, average position, etc.) canbe provided so that advertisers have information to assist them inmaking their ads achieve their goals (e.g. by alerting advertisers thatad creatives should be modified, maximum bid amounts increased ordecreased to achieve or maintain a certain position or preference,etc.).

§4.2.2 Exemplary Billing Methods

FIG. 8 is a flow diagram of an exemplary method 228′ that may be used toeffect billing operation(s) 228. As shown by trigger block 810, if costinformation 229 is available, such information is accepted as indicatedby block 820. Then, as indicated by decision block 830, it is determinedwhether or not there is a condition(s) precedent to billing. That is, ifan advertiser agrees to pay only when its ad is clicked or only whenconversion occurs, a condition precedent (e.g., click-through orconversion) needs to occur before that advertiser is billed, even if itsad was served. If, on the other hand, the advertiser agrees to pay amaximum cost per impression, it can be billed once its ad is served. Ifthere is a condition(s) precedent to billing, as indicated by decisionblock 850, it is determined whether or not this condition(s) is met. Ifnot, the cost information is stored (for later billing, if necessary) asindicated by block 860, and processing returns to the trigger block 810.If, on the other hand, the condition(s) is met, the determined cost canbe billed to the advertiser's account, as indicated by block 840, beforethe method 228′ is left via RETURN node 899. Referring back to decisionblock 830, if there is no condition(s) precedent to billing, thedetermined cost can simply be billed to the advertiser's account, asindicated by block 840, before the method 228′ is left via RETURN node850.

Referring back to trigger block 810, if a confirmation of a result(s) isreceived (e.g., via the result interface operation(s) 260), it isdetermined whether or not a condition(s) precedent to billing, if any,is met as indicated by decision block 870. If not, the method 228′ isleft via RETURN node 899. If, on the other hand, the condition(s)precedent to billing, if any, is met, processing continues to decisionblock 880 where it is determined whether or not a determined costinformation was provided with the result confirmation. If so, thedetermined cost is billed, as indicated by block 840, before the method228′ is left via RETURN node 899. Referring back to decision block 880,if, on the other hand, the determined cost information is not providedin the result confirmation, then (assuming that the determined costinformation was previously stored (Recall block 860.)) stored determinedcost information is retrieved, as indicated by block 890, before thedetermined cost amount is billed as indicated by block 840 and themethod 228′ is left via RETURN node 899.

§4.2.3 Exemplary Apparatus and Data Structures

FIG. 9 is high-level block diagram of a machine 900 that may effect oneor more of the operations discussed above. The machine 900 basicallyincludes a processor(s) 910, an input/output interface unit(s) 930, astorage device(s) 920, and a system bus or network 940 for facilitatingthe communication of information among the coupled elements. An inputdevice(s) 932 and an output device(s) 934 may be coupled with theinput/output interface(s) 930.

The processor(s) 910 may execute machine-executable instructions (e.g.,C or C++ running on the Solaris operating system available from SunMicrosystems Inc. of Palo Alto, Calif. or the Linux operating systemwidely available from a number of vendors such as Red Hat, Inc. ofDurham, N.C.) to effect one or more aspects of the present invention. Atleast a portion of the machine executable instructions may be stored(temporarily or more permanently) on the storage device(s) 920 and/ormay be received from an external source via an input interface unit 930.

In one embodiment, the machine 900 may be one or more conventionalpersonal computers. In this case, the processing unit(s) 910 may be oneor more microprocessors. The bus 940 may include a system bus. Thestorage devices 920 may include system memory, such as read only memory(ROM) and/or random access memory (RAM). The storage device(s) 920 mayalso include a hard disk drive for reading from and writing to a harddisk, a magnetic disk drive for reading from or writing to a (e.g.,removable) magnetic disk, and an optical disk drive for reading from orwriting to a removable (magneto-) optical disk such as a compact disk orother (magneto-) optical media.

A user may enter commands and information into the personal computerthrough input devices 932, such as a keyboard and pointing device (e.g.,a mouse) for example. Other input devices such as a microphone, ajoystick, a game pad, a satellite dish, a scanner, or the like, may also(or alternatively) be included. These and other input devices are oftenconnected to the processing unit(s) 910 through an appropriate interface930 coupled to the system bus 940. However, in the context of theaccounting/billing operation(s) 225, no input devices, other than thoseneeded to accept data, and possibly those for system administration andmaintenance, are needed.

The output device(s) 934 may include a monitor or other type of displaydevice, which may also be connected to the system bus 940 via anappropriate interface. In addition to (or instead of) the monitor, thepersonal computer may include other (peripheral) output devices (notshown), such as speakers and printers for example. Again, in the contextof accounting/billing operation(s), no output devices, other than thoseneeded to output billing information, and possibly those for systemadministration and maintenance, are needed.

FIG. 10 illustrates a relationship among ad identification information1010, keyword(s) 1020, ad content 1030, a maximum cost per result bid1040 and an average cost per result bid 1050. Various data structuresfor associating such information are possible. In one embodiment, the adidentifier 1010 may be unique and therefore used as a “primary key” toassociate various pieces of information.

§4.2.4 Alternatives and Refinements

In the exemplary embodiment of ordering ads described above withreference to FIG. 5, ordering was strictly a matter of relative position(or ad preference) values. However, in one exemplary embodiment,advertisers can define constraints. For example, if an advertiser sets a“top position only” constraint, if their ad does not have the topposition value, it will not be served and ads with lower position (or adpreference) values will move up. Referring to the example provided inFIG. 6, suppose ad C has such a “top position only” constraint. In thisexample, ad C would be dropped and ads E and D would each move up onespot. In this case, the cost for placing ad A second could be droppedfrom $4.00 to $3.00 (ad E's placement value divided by ad A'sperformance parameter=0.030/0.01). In another example, if an advertisersets a “no higher than position three” constraint, if the position valueof their ad is the highest, the ad with the second highest positionvalue will move up to the first position, the ad with the third highestposition value will move up to the second highest position, and the“constrained” ad will move down two positions to the third highestposition. Referring to the example provided in FIG. 6, if ad A had a “nohigher than third position” constraint, ad C would take ad A's secondposition, and ad A would move to the third position. In this case, sincead C could not fall below ad A, but could fall below ad E, its actualcost could remain at $0.75. However, ad A's actual cost could be $3.00(ad E's placement value divided by ad A's performanceparameter=0.030/0.01).

In another alternative, advertisers could also specify a maximummarginal cost. For example, if an ad has an 11% click through rate at atop position and a 10% click through rate at a second position, and ifthe cost is $0.60 per click for the top position and $0.50 per click forthe second position, the nominal cost for an additional click is $1.60($6.60 for 11 clicks versus $5.00 for 10 clicks). The advertiser couldspecify a maximum marginal cost per result bid instead of, or inaddition to, the maximum cost per result bid.

In yet another alternative, advertisers could specify a maximum averagecost per result bid. In such a case, in some instances, the costs billedto the advertiser could be above this maximum average cost per resultbid, so long as the average of all of the costs billed is below themaximum average cost per result bid. In a conservative implementation ofthis embodiment, initial instances would be at or less than the maximumaverage cost per result bid. In this way, a “surplus” could accrue andbe applied to later instances which could be greater than the maximumaverage cost per result bid. In a less conservative implementation ofthis embodiment, initial instances could be above the maximum averagecost per bid, perhaps subject to some limit, under the assumption thatlater costs would bring the average back to, or below, the maximumaverage cost per result bid. In this less conservative implementation,if later costs did not bring the average back to, or below, the maximumaverage cost per result bid by the time of the billing cycle, the amountbilled to the advertiser could be limited such that its maximum averagecost per result bid is not exceeded.

§4.3 OPERATIONS

FIG. 11 is a messaging diagram illustrating exemplary operations of afirst exemplary embodiment of the present invention. As indicated bycommunication 1110, an ad serving operation(s) 230 receives an adrequest from an ad consumer 130. The ad request may include the numberof ads desired, as well as other information. In response to therequest, the ad serving operation 230 may get ordered, relevant ads(Recall, for example, operations 235 and 250 of FIG. 2). As shown bycommunications 1120 and 1130, the ad serving operation(s) 230 may then(i) provide, for each ad determined in response to the request 1110, thead content (or an identifier for obtaining the ad content) and aplacement value to the ad consumer 130, and (ii) provide, for each addetermined in response to the request 1110, ad information (preferablyincluding at least an ad identifier and an ad placement value) to a costdetermination operation(s) 227. In response, the cost determinationoperation(s) 227 may determine, for each ad served, a cost that may,perhaps subject to a condition precedent, be billed to an advertiser. Asindicated by communication 1140, the cost determination operation(s) 227may then store the ad information, for each ad served, (preferablyincluding at least an ad identifier and a determined cost) in storagedevice 229. Further, as indicated by communication 1150, the costdetermination operation(s) 227 may indicate to the billing operation(s)228, that the cost information is available. If billing is subject to acondition precedent (such as a click-through or a conversion forexample), the billing operation(s) will await result information fromthe result interface operation(s) 260 as indicated by message 1160. Ifsuch a condition precedent is met, or is not required (such as if a costis a cost per impression), the billing operation(s) can request andreceive the determined cost stored in storage 229, as indicated bycommunications 1170 and 1180, respectively. The billing operation(s) maythen generate billing information to be charged to the relevantadvertiser(s), as indicated by communication 1190.

FIG. 12 is a messaging diagram illustrating exemplary operations of asecond exemplary embodiment of the present invention. This secondembodiment is useful when billing is subject to a condition precedent.As indicated by communication 1210, an ad serving operation(s) 230receives an ad request from an ad consumer 130. The ad request mayinclude the number of ads desired, as well as other information. Inresponse to the request, the ad serving operation 230 may get ordered,relevant ads (Recall, e.g., operations 235 and 250 of FIG. 2.). As shownby communication 1220, the ad serving operation(s) 230 may then provide,for each ad determined in response to the request 1210, ad information(preferably including at least an ad identifier and an ad placementvalue) to a cost determination operation(s) 227. In response, the costdetermination operation(s) 227 may determine, for each ad to be served,a cost that may, perhaps subject to a condition precedent, be billed toan advertiser. As indicated by communication 1230, the costdetermination operation(s) 227 may then provide ad information(preferably including at least an ad identifier and the determined cost)back to the ad serving operation(s) 230. In response, the ad servingoperation(s) 230 provides, for each ad, the ad content (or an identifierfor obtaining the ad content) and the determined cost information andplacement information, as indicated by communication 1240. In this case,it is assumed that billing is subject to a condition precedent (such asa click-through or a conversion for example). Thus, the billingoperation(s) will await result information from the result interfaceoperation(s) 260 as indicated by message 1250. Here, the communication1250 preferably includes the previously determined ad cost information,in addition to an ad identifier and result information. The resultinformation may be implied, for example from the presence or absence ofa result message. If the condition precedent is met, the billingoperation(s) 228 may then generate billing information to be charged tothe relevant advertiser(s), as indicated by communication 1260.

§4.4 CONCLUSIONS

As can be appreciated from the foregoing disclosure, the advertisingtechniques described above account for the interests of users (e.g.,users of a search engine) by providing them with ads that are ofinterest and that are not easily confused with noncommercial requestedcontent (such as search results), if any. Further, the interests of theadvertisers are well served since users will presumably be more likelyto click-through on ads that are of interest to them. Moreover, thepresent invention allows advertisers to manage their “bidding” forplacing their advertisement easily, while being assured that they won'tpay more than they need to.

The foregoing description of preferred embodiments of the presentinvention provides illustration and description, but is not intended tobe exhaustive or to limit the invention to the precise form disclosed.Modifications and variations are possible in light of the aboveteachings or may be acquired from practice of the invention. Forexample, the accounting and billing operation(s) 225 can be distributedamong various ad consumers 130, rather than operating centrally at thead entry, maintenance and delivery system(s) 120.

1. A computer-implemented method comprising: a) receiving, by a computersystem including at least one computer, a plurality of ads; b)determining, by the computer system, an ad preference value for each ofthe plurality of ads as a function of performance information associatedwith the ad and a maximum cost for a result bid associated with the ad,wherein the performance information associated with the ad is selectedfrom a group consisting of (i) a selection rate of the ad, (ii) aconversion rate of the ad, (iii) an expected selection rate of the ad,and (iv) an expected conversion rate of the ad; c) determining, by thecomputer system, a cost for each of the plurality of ads, the cost beinglower than the maximum cost for a result bid associated with the ad, byi) accepting a next lower ad preference value than that of the ad; ii)accepting the performance information associated with the ad; and iii)determining the cost based on the accepted performance information ofthe ad and the accepted next lower ad preference value such that a valueprovided by the function of the accepted performance information of thead and the determined cost of the ad, is not less than the accepted nextlower ad preference value.
 2. The computer-implemented method of claim 1wherein the ad preference value is a placement value.
 3. Thecomputer-implemented method of claim 1 wherein, for each of theplurality of ads, the performance information associated with the ad isa click-through rate of the ad, and wherein the maximum cost for aresult bid associated with the ad is a maximum cost per click-throughbid.
 4. The computer-implemented method of claim 3 wherein, for each ofthe plurality of ads, the ad preference value is a function of theclick-through rate of the ad and the maximum cost per click-through bidassociated with the ad.
 5. The computer-implemented method of claim 3wherein, for each of the plurality of ads, the ad preference valueincludes a multiplicative product of the click-through rate of the adand the maximum cost per click-through bid associated with the ad. 6.The computer-implemented method of claim 5 wherein the cost of the ad isdetermined by dividing the next lower ad preference value by theclick-through rate of the ad.
 7. The computer-implemented method ofclaim 6 further comprising: d) accepting, by the computer system, resultinformation indicating whether or not click-through on one of theplurality of ads occurred; and e) billing, by the computer system, thecost determined to an advertiser associated with the one of theplurality of ads only if the accepted result information indicates thatclick-through occurred.
 8. The computer-implemented method of claim 1wherein, for each of the plurality of ads, the performance informationassociated with the ad is a conversion rate of the ad, wherein aconversion includes a purchase, and wherein the maximum cost for aresult bid associated with the ad is a maximum cost per conversion bid.9. The computer-implemented method of claim 8 wherein, for each of theplurality of ads, the preference value of the ad is a function of theconversion rate of the ad and the maximum cost per conversion bidassociated with the ad.
 10. The computer-implemented method of claim 8wherein, for each of the plurality of ads, the preference value of thead includes a multiplicative product of the conversion rate of the adand the maximum cost per conversion bid associated with the ad.
 11. Thecomputer-implemented method of claim 10 wherein the cost of the ad isdetermined by dividing the next lower ad preference value by theconversion rate of the ad.
 12. The computer-implemented method of claim11 further comprising: d) accepting, by the computer system, resultinformation indicating whether or not conversion of one of the pluralityof ads occurred; and e) billing, by the computer system, the costdetermined to an advertiser associated with the one of the plurality ofads only if the accepted result information indicates that conversionoccurred.
 13. The computer-implemented method of claim 1 furthercomprising: d) encoding, by the computer system and for each of theplurality of ads, the cost determined; and e) serving, by the computersystem and for each of the plurality of ads, the encoded cost with thead.
 14. The computer-implemented method of claim 1 wherein each of theplurality of ads is associated with a group of one or more keywords. 15.The computer-implemented method of claim 1 wherein, for each of theplurality of ads, the maximum cost for a result bid is a maximum averagecost per result associated with the ad.
 16. The computer-implementedmethod of claim 1 wherein, for each of the plurality of ads, the maximumcost for a result bid is a maximum cost for an incremental result bidassociated with the ad.
 17. Apparatus comprising: a) one or moreprocessors; b) at least one input device; and c) one or more storagedevices storing processor-executable instructions which, when executedby one or more processors, perform a method of: 1) receiving, via the atleast one input device, a plurality of ads, 2) determining an adpreference value for each of the plurality of ads as a function ofperformance information associated with the ad and a maximum cost for aresult bid associated with the ad, wherein the performance informationassociated with the ad is selected from a group consisting of (i) aselection rate of the ad, (ii) a conversion rate of the ad, (iii) anexpected selection rate of the ad, and (iv) an expected conversion rateof the ad, 3) determining a cost for each of the plurality of ads, thecost being lower than the maximum cost for a result bid associated withthe ad, by i) receiving as input, for each of the plurality of ads, anext lower ad preference value than that of the ad, and the performanceinformation associated with the ad, and ii) determining the cost basedon the performance information of the ad and the accepted next lower adpreference value such that a value provided by the function of theperformance information of the ad and the determined cost of the ad, isnot less than the next lower ad preference value.
 18. The apparatus ofclaim 17 wherein the ad preference value is a placement value.
 19. Theapparatus of claim 17 wherein the performance information associatedwith the ad is a click-through rate of the ad, and wherein the maximumcost for a result bid associated with the ad is a maximum cost perclick-through bid.
 20. The apparatus of claim 19 wherein, for each ofthe plurality of ads, the ad preference value is a function of theclick-through rate of the ad and the maximum cost per click through bidassociated with the ad.
 21. The apparatus of claim 19 wherein, for eachof the plurality of ads, the ad preference value includes amultiplicative product of the click-through rate of the ad and themaximum cost per click through bid associated with the ad.
 22. Theapparatus of claim 21 wherein the act of determining the cost of the adis determined by dividing the next lower ad preference value by theclick-through rate of the ad.
 23. The apparatus of claim 22 wherein theact of receiving as input further receives as input, result informationindicating whether or not click-through on one of the plurality of adsoccurred, the method further including 4) billing the cost determined toan advertiser associated with the one of the plurality of ads only ifthe result information indicates that click-through on the ad occurred.24. The apparatus of claim 17 wherein the performance informationassociated with the ad is a conversion rate of the ad, wherein aconversion includes a purchase, and wherein the maximum cost for aresult bid associated with the ad is a maximum cost per conversion bid.25. The apparatus of claim 24 wherein, for each of the plurality of ads,the preference value of the ad is a function of the conversion rate ofthe ad and the maximum cost per conversion bid associated with the ad.26. The apparatus of claim 24 wherein, for each of the plurality of ads,the preference value of the ad includes a multiplicative product of theconversion rate of the ad and the maximum cost per conversion bidassociated with the ad.
 27. The apparatus of claim 26 wherein the act ofdetermining the cost of the ad is determined by dividing the next lowerad preference value by the conversion rate of the ad.
 28. The apparatusof claim 27 wherein the act of receiving as input further receives asinput, result information indicating whether or not conversion of one ofthe plurality of ads has occurred, the method further including 4)billing the cost determined to an advertiser associated with the one ofthe plurality of ads only if the accepted result information indicatesthat conversion on the ad occurred.
 29. The apparatus of claim 17wherein each of the plurality of ads is associated with a group of oneor more keywords.
 30. The apparatus of claim 17 wherein, for each of theplurality of ads, the maximum cost for a result bid associated with thead is a maximum average cost per result.
 31. The apparatus of claim 17wherein, for each of the plurality of ads, the maximum cost for a resultbid is a maximum cost for an incremental result bid associated with thead.
 32. A computer-implemented method comprising: a) determining, by acomputer system including at least one computer and for each of aplurality of ads, an ad preference value as a function of performanceinformation associated with the ad and a maximum cost for a result bidassociated with the ad, wherein the performance information associatedwith the ad is selected from a group consisting of (i) a selection rateof the ad, (ii) a conversion rate of the ad, (iii) an expected selectionrate of the ad, and (iv) an expected conversion rate of the ad; b)ordering, by the computer system, the plurality of ads using thedetermined ad preference values; c) controlling, by the computer system,a serving of the plurality of ads using the ordering such that arendering of the ads will be controlled; and d) determining, by thecomputer system, a cost lower than the maximum cost associated with oneof the plurality of ads, by i) accepting an ad preference value ofanother of the plurality of ads having a next lower preference value tothe preference value of the one of the plurality of ads, ii) acceptingthe performance information associated with the one of the plurality ofads, and iii) determining the cost by dividing (1) the accepted an adpreference value of the other of the plurality of ads having a nextlower preference value to the preference value of the one of theplurality of ads by (2) the performance information associated with theone of the plurality of ads.